cc @jroesch @wweic the CI problem could relate to limitations in VM

operator fusion tests are heavily dependent on concat being fusible as injective. If we want to make concat opaque, corresponding updates to fusion tests are needed.

If this is indeed a temporary solution, I suggest just disable fusion tests that involve concat.

I have a patch for the VM issue which is almost complete, will post tomorrow sometime.

Will this change affect the performance of concat for cpu？

Will this change affect the performance of concat for cpu？

Previously ssd_mobilenet1.0_512 cost 1.4437s on arm CPU now it costs 1.4545s.

I tested on x86 cpu with a set of gluoncv models. This change doesn't affect the performance.

This fix solves 10-15 min hanging issue for the first inference
But it causes 20% performance degradation for the second and consequent inferences on GPU.
I tested it for GluonCV ssd_512_mobilenet1.0_voc model on Mali GPU OpenCL (RK3399)
The second and consequent inferences take:
before the fix: 420ms
after the fix: 520ms

This fix solves 10-15 min hanging issue for the first inference
But it causes 20% performance degradation for the second and consequent inferences on GPU.
I tested it for GluonCV ssd_512_mobilenet1.0_voc model on Mali GPU OpenCL (RK3399)
The second and consequent inferences take:
before the fix: 420ms
after the fix: 520ms

The performance reported by tvm evaluator with hanging issue was 470 ms on Mali GPU RK3399. But the hanging time is more than 10 minutes which composes most of the end-to-end time. And by this fix, the hanging issue is alleviated, and performance evaluated by tvm time evaluator is 517 ms. You are more than welcome to fix the hanging issue without performance lost.

operator fusion tests are heavily dependent on concat being fusible as injective. If we want to make concat opaque, corresponding updates to fusion tests are needed.

If this is indeed a temporary solution, I suggest just disable fusion tests that involve concat.

@tqchen Shall we disable fusion tests that involve concat?

@apivovarov, making Concat opaque allows you to write fast schedules specially for concat, for instance, https://github.com/dmlc/tvm/blob/master/topi/python/topi/x86/injective.py#L53 for x86 concat schedules. Note that the concat schedule is only guaranteed to be used only when concat is an opaque or output_elem_fusable op. The default injective schedule would be used when concat is an injective op and fused together with other injective ops.

Some related followup thoughts https://discuss.tvm.ai/t/explore-optimizations-for-concat/2435/7

Close this for now due to inactive status, @sxjscience has volunteered to continue working on the thread https://discuss.tvm.ai/t/explore-optimizations-for-concat/2435/7

@tqchen I'm still working on that. Need to wait for some more time...